In order to improve the efficiency of power transmission line inspection by Unmanned Aerial Vehicle (UAV), a new method was proposed for detecting broken transmission lines and defects of foreign body based on the perception of line structure. The transmission line image acquired by UAV was easily influenced by the background texture and light, the gradient operators of horizontal and vertical direction which can be used to detect the line width were used to extract line objects in the inspection image. The study on calculation of gestalt perception of similarity, continuity and colinearity connected the intermittent wires into continuous wires. Then the parallel wire groups were further determined through the calculation of parallel relationship between wires. In order to reduce the detection error rate, spacers and stockbridge dampers of wires were recognized based on a local contour feature. Finally, the width change and gray similarity of segmented conductor wire were calculated to detect the broken part of wire and foreign object defect. The experimental results show that the proposed method can detect broken wire strand and foreign object defect efficiently under complicated backgrounds from the transmission line of UAV images.